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ABSTRACT 



A high definition television system that is characterized 
by low transmission bandwidth is achieved by in a man- 
ner removing much of the redundancies in the signal, 
efficiently encoding the remaining signals, and transmit- 
ting the encoded signal in a manner that is most compat- 
ible with the applicable standards. Specifically, the tele- 
vision signal is encoded by developing motion vectors 
that describe the best motion estimate of the image to be 
transmitted, by developing motion estimation error 
signals, by encoding these error signals within the same 
bandwidth as occupied by standard NTSC signals and 
by transmitting the encoded error signals during peri- 
ods that correspond to the active scan intervals of the 
NTSC TV signal. The motion vectors themselves, to- 
gether with video and control signals, are transmitted 
during the NTSC retrace period. 
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cant difference between the two blocks, the difference 
HIGH DEFINITION TELEVISION is encoded and sent to the decoder so that the pels of the 

ARRANGEMENT EMPLOYING MOTION current block may be more accurately recovered. Cod- 

COMPENSATED PREDICTION ERROR SIGNALS ing of this difference is typically performed by means of 

5 the "Discrete Cosine Transform" (DCT). It is a mea- 
BACKGROUND OF THE INVENTION sure of energy. 

This invention relates to high definition television. The amount of coded information that is generated 
More particularly, this invention relates to a method by the above procedure is variable. It can be appreci- 
and apparatus for encoding and decoding video signals &ted f for example, that image changes that do not corre- 
for HDTV. 10 spond to a uniform translation, or motion, of the image 

This application is related to a number of applications may require substantial encoding to describe the devia- 
filed on even date herewith, titled: "A High Definition tion of a block from its best translated replica. On the 
Television Arrangement With Signal Selections other hand, when the image does not change between 
Adapted to the Available Transmission Capacity", successive frames, then there is a minimal amount of 
"PAM Signal Modulation With Mappings to Improve 15 information that needs to be encoded. To accommodate 
Utilization of Available Transmission Capacity", "A these potentially wide variations in the amount of code 
High Definition Television Arrangement Including that needs to be transmitted, typical encoders include a 
Noise Inimunity Means", and M A Television Signal memory at the output, to serve as a buffer. 
Arrangement Where Selected Signals are Encoded The buffer is not a panacea, however. For a given 
Digitally". 20 transmission rate, when an excessive volume of data is 

Video signals typically originate from video cameras. generated, there is always a danger that the FIFO 
The bandwidth of video signals is quite substantial and, would overflow. When it does, coding must stop until 
consequently, practitioners in the art have tried to re- the transmission channel can empty the FIFO suffi- 
duce the bandwidth of these signals without unduly cientl t0 it new data t0 be insemd 
degrading the images. Typically, to reduce bandwidth, 25 M{ q{ ^ ^ ^ ^ ^ ^ ^ ^ 
the video signals are encoded and redundancies in the coding and decoding aspec ts of reducing the bandwidth 

encoded signals are extracted and deleted. Different ^r«vJV\/*; i w„* * a 1 V S *u *u r 

* a * *u - c u >.ac of the TV signal, but none deal explicitly with the for- 

tecnmques are used in the art. Some are better suited for „. P ,f ' , . r * 

still images, while others are better suited for moving ° f the S g » al ^ P «ion for transmission, 

images. One of the techniques for reducing the band- 30 ^ n *> f h * h definition television both the 

width of moving images is generally referred to as mo- h ™ d ™ d * a forma " in S P"> bl «™ must be solved 
tion compensated predictive coding. and u the difficulties are even greater than in connection 

In conventional motion compensated predictive cod- Wlth conventional TV signals because the desired signal 
ing, each video frame is first partitioned into square compression is even greater, and because the require- 
blocks of picture elements (pels); such as blocks of 8 pels 35 for 8 more authentic representation of the original 
by 8 pels. Each block is coded, in turn, and the devel- una « e m morc stringent, 
oped encoded sequence is transmitted over a communi- SUMMARY OF THE INVENTION 

cations channel to a decoder. The communications 

channel may be, or may include, a storage element. A hi « h definition television system that is character- 
Following the partitioning step in the encoding process, 40 bv low transmission bandwidth is achieved by in a 
a determination is made as to whether or not the pels of manner removing much of the redundancies in the sig- 
the block have changed significantly compared with the nal » efficiently encoding the remaining signals, and 
previous frame. If not, an indicator signal is sent which transmitting the encoded signal in a manner that is most 
signifies to the decoder that it needs to merely repeat compatible with the applicable standards. Specifically, 
the pels of that block from the previous frame to obtain 45 tne television signal is encoded by developing motion 
the pels for the current block. This step is known as vectors that describe the best motion estimate of the 
"Conditional Replenishment*'. If the pels have changed image to be transmitted, by developing motion estima- 
since the previous frame, an attempt is made to deter- tion error signals, by encoding these error signals within 
mine the best estimate of motion that is occurring in the tne standard NTSC band and by transmitting the en- 
bloclc This is frequently done by a "Block Matching 50 coded error signals during periods that correspond to 
Motion Estimation** technique wherein the pels of the the active scan intervals of the TV signal. The motion 
current block are successively compared with various vectors themselves, together with video and control 
small shifts of the corresponding block in the previous signals, are transmitted during the retrace period, 
frame. The shift that gives the best match is deemed to _ nP « rp TPTinK nT7 TWP np A WT ™ 

be the "best estimate** of the displacement in the block's 55 BRIEF DESCRIPTION OF THE DRAWING 
image between frames, and the amount of this shift, FIG. 1 depicts the conventional television scan ar- 
called the "Motion Vector", is selected and sent to the rangement; 

decoder. FIG. 2 presents a block diagram of the signal encoder 

The "best estimate" is, of course, the estimate that portion of a video transmitter; 
yields the smallest difference signal between the image 60 FIG. 3 presents a block diagram of a receiver com- 
in the current block and the image in the shifted block porting with the signals developed by the FIG. 2 trans- 
of the previous frame. This difference signal forms the mitter; 

error signal. When the error signal is sufficiently small, FIG. 4 is a detailed block diagram of the motion 
an indicator signal is sent to the decoder, which merely compensator circuit in the FIG. 2 transmitter; 
causes the pels of the shifted block from the previous 65 FIG. 5 illustrates the concept of motion compensa- 
frarae to be repeated for the pels for the current shifted tion; 

block. Such blocks are said to have been successfully FIG. 6 develops one structure of the two-dimensional 
"Motion Compensated' 1 . However, if there is a signifi- transformation circuit in the FIG. 2 arrangement; 
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FIG. 7 presents a pictorial view of the subbands de- 
veloped in the transmitter and the vectors that define 
the subband signals; 

FIG. 8 depicts a PAM signal and its utilization of the 
available transmission capacity; 5 

FIG. 9 presents a portion of an encoding look-up 
table that is useful in encoding two samples into one; 

FIG. 10 presents the block diagram of one arrange- 
ment of the encoder within the FIG. 2 transmitter; 

FIG. 11 presents the block diagram of another ar- 10 
rangement of the encoder within the FIG. 2 transmitter; 

FIG. 12 is a block diagram of a transmitter which 
includes leak and a controllable gain feature; and 

FIG. 13 is a block diagram of a receiver that com- 
ports with a transmitter of FIG. 12. 15 

DETAILED DESCRIPTION 

FIG. 1 depicts a conventional TV raster. The picture 
is developed by scanning the image one line at a time. 
Each scan line (21) is followed by a horizontal retrace 20 
22 and the last line of the frame is followed by a vertical 
retrace 23 which includes a vertical sync pulse embed- 
ded in the retrace interval. Conventional TV includes 
an "interlace" concept, but for the purposes of this 
description it is of no significance. It may be noted, 25 
however, that the aspect ratio of HDTV, which is ex- 
pected to be 16 to 9, is different from that of conven- 
tional TV. This fact also has very little significance to 
this description. 

FIG. 2 depicts a general block diagram of a transmit- 30 
ter following the principles of our invention. Block 110 
is responsive to an incoming frame and to a frame 
within buffer 160. It develops low frequency coeffici- 
ents (LL coefficients) and motion vectors which are 
applied to formatter 170 and translation circuit ISO, and 35 
motion-compensated prediction error signals which are 
applied to transformer block 120. Block 120 performs a 
transformation on the applied signals (basically a time 
domain to frequency domain transformation) and ap- 
plied the results to encoder block 130. Encoder 130 40 
develops a greatly reduced set of signals, in the form of 
discrete amplitude error signals and vector index sig- 
nals, which, in combination, represent the signals cre- 
ated in block 120, and forwards the signals streams to 
formatter 170 and decoder 159. The signals developed 45 
in encoder 130 are selected and arranged to fill the 
available transmission capacity with the information 
that is most important for an authentic recreation of the 
original image. 

In the feedback portion of the transmitter, decoder 50 
159 is responsive to the error signals and the vector 
index signals of encoder 130. Its function is to recover 
the frequency coefficients that were selected and en- 
coded in block 130. Its output is applied to transform 
circuit 161, and its function is to reconstitute, as best it 55 
can, the input signals of transformation block 120. The 
output of block 161 is added in element 180 to a motion 
compensated estimate of the frame and the sum is 
placed in buffer 160. The motion compensated estimate 
is developed in translation circuit 150 which is respon- 60 
sive to buffer 160 and to the motion vectors. 

Formatter 170 combines the information supplied by 
blocks 130 and 110 with audio information associated 
with the frame, and creates a signal in a format that is 
suitable for whatever transmission medium is employed 65 
by the user. In accordance with one aspect of this inven- 
tion, the error signals developed by encoder 130 are 
organized to fit within, and are formatted to be in, the 



line scans interval of the frame. All of the other signals 
applied to formatter 170 are encoded into the retrace 
interval of the frame. 

The receiver that corresponds to the transmitter of 
FIG. 2 is shown in FIG. 3. It includes a separation block 
190 which recovers the audio signals, the LL and mo- 
tion vector signals of block 110 and the output signals of 
encoder 130. The signals corresponding to the output 
signals of encoder 130 are applied to decoder block 163. 
Like decoder block 159, block 163 recovers the fre- 
quency coefficients selected and encoded in block 130. 
The output is applied to transform circuit 155 which 
reconstitutes, as best it can, the input signal of transfor- 
mation of block 120. Concurrently, the motion vectors 
developed in block 110 are applied to translator circuit 
164 which modifies the output of buffer 165 in accor- 
dance with those vectors. The result is applied to adder 
185 which sums the output signals of blocks 155 and 164 
and applied the results of buffer 165. Buffer 165 contains 
the EL video frame information of the receiver (which 
is the image, minus the portion represented by the LL 
coefficients). Concurrently with the processing in ele- 
ment 155 and 185, the LL coefficients of block 190 are 
applied to transform block 188 which develops the LL 
image of the receiver. The LL image is added to the LL 
image in element 186, to form the final receiver frame. 
That frame is processed and displayed, and the associ- 
ated audio is processed and converted to sound. 

The details of motion compensation block 110 are 
shown in FIG. 4. Therein, the incoming frame is con- 
nected to buffer 102 and to two-dimensional low pass 
filter 103 coupled to sub-sampler 115. Filter 103 con- 
tains a conventional low pass filter 104, a buffer 105 for 
rearranging the data, a conventional low pass filter 106 
responsive to buffer 105, and a buffer 107 for a second 
rearranging of the data. Filter 103 thus develops two-di- 
mensional low frequency coefficients (LL coefficients) 
which are subsampled in 115 and applied to formatter 
170. The subsampled output of filter 103 is also applied 
to interpolater 108 which develops an LL image corre- 
sponding to the LL coefficients. While the LL coeffici- 
ents and the LL image are developed, the applied image 
frame is delayed in buffer 102. The delayed image and 
the LL image are applied to subtracter 109, which sub- 
tracts the LL image from the applied image to yield an 
image where the low spatial frequencies are missing 
(i.e., the LL image). The LL frame output of subtracter 
109 is applied to frame buffer 111, to motion vectors 
generator block 112 and to adder 114. As the LL frame 
is stored in buffer 111, the previously stored LL frame 
is extracted from buffer 111 and applied to motion vec- 
tors generator block 112. Motion vectors generator 112 
operates on non-overlapping blocks of the image. These 
blocks can be of any size, but the size that we employ in 
connection with the HDTV embodiment described 
therein is 36 by 32 pixels (36 pixels in a row, 32 rows). 
For each selected block, a selected neighborhood of the 
block is searched in the previous frame (found in buffer 
111) for a block of pixels that most approximates the 
block in question. In this manner, a motion vector is 
obtained that points to an area of the image in buffer 111 
that best approximates the block in question. 

The search process, to a single pixel accuracy, is 
perfectly conventional. To improve performance, our 
motion vectors are computed to J pixel accuracy. This 
is accomplished by first developing a motion vector to 
1 pixel accuracy and then interpolating in the neighbor- 
hood pointed to by the motion vector between rows 
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10 



15 



20 



30 



and between pixels in the rows of the previous block to 
determine whether subpixel motion vector specification 
is called for. 

To illustrate, FIG. 5 depicts a region 31 in frame K 
and a block of pixels 30 in frame K+ 1. As can be seen 
from a review of the two, block 30 of frame K+ 1 may 
be described quite accurately by stating that it corre- 
sponds to the region in frame K that is shifted with 
respect to the position of block K by two pixels up- 
wards and three pixels to the right. 

The motion vectors of generator block 112 are ap- 
plied in FIG. 4 to translator 113. The other input of 
translator 113 is the output signal of buffer 160. As 
mentioned above, the output of buffer 160 represents 
the previous EE frame as it is known to the receiver. 
The motion vectors are compared in translator 113 with 
the image of buffer 160 to form a predicted motion- 
compensated LL frame. Typically, this predicted frame 
does not faithfully represent the LL frame. Almost al- 
ways there are errors in the predicted frame. To ascer- 
tain the position and intensity of those errors, the mo- 
tion-compensated LL frame of translator 113 is applied 
to subtracter 114, wherein it is compared to the LL 
frame signal itself. The output of subtracter 114 is a , 
frame of motion-compensated prediction error signals. 25 
Those signals are applied to transformer block 120. 

Transformer block 120 maps the information to the 
frequency domain. Although there are many ap- 
proaches for such a mapping, the approach chosen here 
involves 16 FIR filters, as depicted in FIG. 6. Specifi- 
cally, the input of block 120 is applied to 8 "horizontal" 
filters 121. Each filter spans 64 pixels and develops a 
coefficient that represents the intensity of signals in a 
chosen frequency band. The filters are termed "hori- 
zontal" because the input pixels to a filter are taken from 35 
successive pixels on the same line scan of the frame, and 
because successive coefficients are obtained by sliding 
the filter horizontally. The horizontal shift is by 8 pixels. 
Each developed coefficient is associated with the pixel 
that is at the center of the neighborhood defined by the 40 
64 pixels, and thus, each line scan of M pixels produces 
M/8 coefficients in each of the 8 FIR filters (assuming 
that something is done at the edges of the scan lines — 
such as creating pixels to satisfy for the filter needs). 
The frequency bands that are evaluated by the 8 filters 45 
are chosen, or controlled, by the coefficients that are 
incorporated within the FIR filters. The outputs of the 
"horizontal" niters are appropriately rearranged with 
the aid of buffer 122 and applied to 8 "vertical" filters 
123, which may be identical to the "horizontal" filters 
and which process the signals in the identical fashion. 
The overall effect that is created is that of two-dimen- 
sional filtering. Description of block oriented two-di- 
mensional filtering can be found, for example, in U.S. 
Pat. No. 4,829,465 issued May 9, 1989. 

The output of transformer block 120 can be viewed as 
a collection of 64 subband frames, as depicted in FIG. 7. 
Each subband frame defines a subband with N/8 rows 
of coefficients and M/8 coefficients per row, where N 
and M are the numbers of rows and pixels per row in the 60 
image frame. As can also be observed in FIG. 7, corre- 
sponding coefficients in the subbands can be grouped to 
form vectors, such as vector 35. The elements of such 
vectors all stem from a common set of motion-compen- 
sated prediction error signals (of block 110). Alterna- 
tively, groups of coefficients in each subband (such as 
blocks comprising 2 rows of 4 coefficients) can be com- 
bined to form elements of a vector. This is illustrated by 



50 



55 



65 



vector 36. In accordance with this invention, when 
vectors such as vector 36 are employed, the 64 subband 
frames may be represented by MN/512 vectors. 

It is recognized that there are more efficient and less 
efficient ways of communicating the information con- 
tained in the 64 subband frames. The challenge, of 
course, is to select a more efficient approach rather than 
a less efficient one. It is also recognized that some gran- 
ularity may be introduced, and some information in the 
frequency domain may be ignored without undully 
damaging the image quality. Based on these recogni- 
tions, the function of encoder block 130 is to identify the 
most important image information contained in the 64 
subband frames and to pack this information in the 
available transmission capacity. This may be accom- 
plished in three steps. In the first step of such a three 
step approach, a selection is made to transmit some 
information and to simply ignore some information. In 
the second step, the information to be transmitted is 
approximated by choosing to transmit from a restricted 
lexicon of signals. The lexicon restriction results in 
lower demands on the digital transmission capacity. In 
the third step, the information is packed within the 
available transmission capacity. 

Having mentioned "transmission capacity", it makes 
sense at this point to describe what the* available trans- 
mission capacity is. 

Referring to FIG. 1, the line scans, the horizontal 
retrace and the vertical retrace periods are all directly 
related to the operation of the TV screen. They need 
not relate necessarily to the transmission of information 
to the TV receiver, as long as some means are provided 
to synchronize the screen to the received frame infor- 
mation. Hence, subject to the synchronization require- 
ment, the time interval corresponding to the sum of 
those periods can be devoted entirely to the transmis- 
sion of frame information. 

In a terrestrial transmission environment, each TV 
channel is allocated a specific bandwidth. Typically, at 
the transmitter's end the TV signal is band limited and 
modulated (AM) onto a carrier. To minimize interfer- 
ence between TV channels, a "taboo" guard band sepa- 
rates adjacent TV channels, where no signals should be 
present. The "taboo" bands are needed because the 
band limiting of the baseband signals is not absolute and 
because there are always nonlinearities in the transmit- 
ter. Nonlinearities cause harmonic spillover. Even low 
levels of interference are often objectionable because 
they create "ghosts" in the interfered channel. The 
human eye is quite adept at detecting these patterns. 

The need for reducing interference is greatest during 
the line scan. During the retrace periods, in contradis- 
tinction, much greater interference can be tolerated. At 
such intervals whatever interference is introduced 
needs to be limited only to the point of insuring that the 
interfering signal does not cause a loss of synchroniza- 
tion. In accordance with one aspect of this invention, 
interference is maintained at an acceptably low level by 
limiting the bandwidth of the information sent during 
the line scan intervals to the assigned channel's fre- 
quency band. During retrace, the bandwidth of the 
transmitted information may be allowed to expand into 
the "taboo" channel. More specifically, in accordance 
with the principles of this invention, during the line scan 
intervals we send motion-compensated prediction error 
coefficients that are selected to fit within the allocated 
signal bandwidth. All other information is sent during 
the retrace intervals. 
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Since the energy in the error signals represents the 
deviation of the predicted image from the true image, it 
is desirable to transmit as much of the energy repre- 
sented by the motion-compensation prediction error 
coefficients as is possible. The largest amount of energy 5 
that the transmitter can impart to a receiver corre- 
sponds to the largest swings in the carrier's amplitude. 
Such "largest'* carrier amplitude results in high positive 
and negative voltage level swings at the receiver. The 
product represented by the maximum RMS level of the 10 
receiver's voltage, times the interval during which that 
level is maintained, shown in FIG. 8 by area 99. 

When signal power is not considered, the objective is 
to pack as many of the motion-compensated prediction 
error coefficients in the time interval of area 99. The 15 
motion-compensated prediction error coefficients, 
which are analog in value, can be represented by analog 
valued samples and the samples can be concatenated to 
form a step-like analog signal. The number of samples 
that can be packed within the line scan interval is lim- 20 
ited by the permissible bandwidth of the modulating 
analog signal and by the ability of the receiver to re- 
solve the signals in the time domain. The utilization of 
area 99, under such circumstances, may be as depicted 
by curve 98 in FIG. 8. More specifically, the utilization 25 
is depicted by the area below area 98, which represents 
the RMS values of the analog samples. 

From the above, it appears that a more efficient utili- 
zation of the transmission models can be achieved by 
increasing the area under curve 98 and reducing the 30 
area above it. This can be accomplished by encoding 
the error signals appropriately. The resulting signal may 
then be like the one depicted by curve 97 in FIG. 8. 
Each level in the signal of curve 97 represents either 
one or more signal pulses. The process of combining a 35 
number of signals to form one signal is a "many to one" 
mapping. 

The combining of digitized signals can, for example, 
be quite straight forward. When a pair of like-signed 
signals has amplitude values lower than a selected 40 
value, such as the square root of the highest permissible 
amplitude, then the pair of signals may be combined to 
form a single digitized signal. The value of the resulting 
single signal may be dictated by a lookup table like the 
one depicted in FIG. 9. For example, when a first signal 45 
has the value 1 1 and a second signal has the value 3, the ' 
combined signal developed in accordance with the 
FIG. 9 table has the value 41. When converted to PAM 
format, a pulse of height 41 is sent to the receiver to- 
gether with an indication that the pulse represents the 50 
combination of two signals. Upon receipt of such an 
indication and the value 41, the receiver accesses a 
similar look-up table and derives therefrom the two 
PAM pulses. 

When a "one to many" mapping results in a signal 55 
with a specific analog level, as in the case above, great 
care must be taken in the encoding algorithm because 
transmission noise is always a factor. Specifically, the 
encoding algorithm must be such that a received signal 
that includes a small deviation from the transmitted 60 
level should not decode to signals that are markedly 
different from the signals that formed the transmitted 
level. It may be noted that the coding in the FIG. 9 table 
is so arranged. For example, a reception of level 10 
when level 11 was transmitted does not cause an error 65 
in the decoded level of the second PAM pulse and 
changes the level of the first decoded PAM pulse by 
only one. The reception of level 16 when level 15 was 
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transmitted causes no errors in the decoded level of the 
first PAM pulse and only an error of one level in the 
decoded level of the second PAM pulse. 

The above describes a "many to one" mapping that 
aims to make the overall PAM signal out of the trans- 
mitter as large as possible. Actually, a similar conse- 
quence results from a "one to many" mapping where 
large digitized signals are encoded into two or more 
smaller signals. The encoding algorithm may be quite 
simple because the decoding process may simply be a 
combining of the constituent pulses. The "one to many" 
mapping improves utilization of area 99 (as does the 
"many to one" mapping) because it reduces the dy- 
namic range of the signal and permits a more effective 
gain control mechanism. 

Whereas the FIG. 9 arrangement depicts an approach 
for combining two digitized signals, it should be clear 
that three (or more) digitized signals can be combined in 
a similar manner with a table (of corresponding dimen- 
sionality) that follows the concepts of FIG. 9. 

When combining and splitting of the prediction error 
signals is not employed, the maximum number of PAM 
pulses that can be transmitted in region 99 is fixed (by 
the transmission bandwidth). When combining and 
splitting is employed, the maximum number of pulses 
that can be transmitted is no longer fixed. Rather, it 
depends on the signal characteristics (although one 
form of mappings negates the effects of the other form 
of mappings). Still, experience with the transmission of 
various images gives some indication as to the percent- 
age of digitized signals that can be combined, and that 
percentage provides an indication of the maximum 
number of signals that can be combined and transmitted 
to the receiver. 

Returning to the detailed description of the FIG. 2 
transmitter, FIG. 10 presents a detailed block diagram 
of encoder 130 for the aforementioned three step ap- 
proach. For the selection step, we chose to evaluate the 
subband frames of FIG. 7 through vectors 36. Specifi- 
cally, it was chosen to combine the energies in the 8 
coefficients of each element of a vector (the 8 coeffici- 
ents in the 2 by 4 array of coefficients in a subband) and 
to compare the combined value to a threshold, thereby 
developing binary values that reflect the comparison 
results. Stepping through the subband frames in a non- 
overlapping fashion yields a set of vectors with 1 and 0 
element values. In FIG. 10, buffer 131 stores the 64 
subband coefficients, outputs groups of 8 coefficients 
and stores them in register 132. Combiner 133 develops 
a measure of the energy in the 8 coefficients and applies 
the results to subtracter 134. Subtracter 134 develops a 
1 output when the combiner signal is greater than the 
threshold, and a 0 output otherwise. This binary output 
is stored in buffer 135. Buffer 135 stores 64 bit vectors. 
Each vector relates to a block of 8 coefficients in the 64 
subbands. A 1 in buffer 135 of the subbands suggests 
that the coefficients that produced the 1 should be trans- 
mitted, and a 0 suggests that the coefficients that pro- 
duced the 0 need not be transmitted. 

The threshold value applied to element 134 can be 
fixed or variable. It should not be set to a value that is 
so high that an insufficient number of coefficients are 
chosen to be transmitted. That would cause some trans- 
mission capacity to be unused. It should also not be so 
low that many more coefficients are selected for trans- 
mission than the available transmission capacity ccfuld 
handle. Such a selection would unnecessarily burden 
the processing equipment. FIG. 8 provides means for 
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allowing the threshold to be set adaptively. Processor 
138 has access to the coefficients in buffer 131. Know- 
ing the transmission capacity, it sorts the coefficients 
(based on the energy of the coefficients) and counts 
from the sorted list the number of PAM pulses that 5 
would be transmitted. When the transmission capacity 
is exhausted, the energy level of the last-accepted coef- 
ficient becomes the applied threshold. 

The assumption implied in deciding that savings will 
accrue when selected coefficients are not transmitted is 10 
that it takes more transmission capacity to transmit 
information which can be ignored, than to transmit the 
additional information necessary for identifying the 
information that is being transmitted. The assumption 
holds true only if the number K of such identifying 15 
information packets (that number being effectively con- 
trolled by the threshold level) and the capacity C re- 
quired for identifying each such packet is such that the 
product KC (which is the capacity needed to identify 
what is transmitted) is less than the capacity saved by 20 
not transmitting the ignored information. This suggests 
that it is important to reduce C as much as possible. 

The simplest way to identify the coefficients that are 
being transmitted and the coefficients that are not being 
transmitted is to transmit the 64 bit vector of Is and 0s 25 
in buffer 135. Keeping the admonition to reduce C in 
mind, we discovered that image quality may be main- 
tained while reducing the number of possible combina- 
tions from 2 M to 2 8 , or 256. A reduced set of possible 
combinations permits one to define each possible combi- 30 
nation of 64 bits with only 8 bits. This mapping of 64 bits 
to 8 bits is achieved by creating a codebook table with 
256 entries. Each entry maintains one of the 64 bit com- 
binations (one codebook vector) and the codebook vec- 
tor is identified by the address (index of the codebook 35 
vector) of the combination in the table. When such a 
codebook is employed} it becomes necessary to judi- 
ciously replace each of the combinations contained in 
buffer 135 with a codebook vector that best represents 
the replaced combination. This is the second step of 40 
encoder 130. 

It is clear that, when considering a particular combi- 
nation of Is and Os in buffer 135, selecting any codebook 
vector that specifies a different combination of Is and Os 
would result in the transmission of at least some coeffi- 45 
cients with lower values than the threshold. That is, 
some high level coefficients that produced a 1 in buffer 
135 may not be sent, and some low level coefficients 
that produced a 0 might be sent. Still, having decoded 
to replace the full set of possible vectors with vectors 50 
from a limited set, it appears beneficial, at this level of 
optimization, to send the vector from the codebook that 
is a) most like the vector it replaces and b) transmits the 
most combined prediction error coefficients energy. 

In FIG. 10, block 136 contains a codebook of 256 55 
vectors of 64 bits each. Processor 137 is responsive to 
the codebook, to buffer 135, and to buffer 131. In accor- 
dance with one approach of this invention and the 
above-described beneficial choices, processor 137 iden- 
tifies the information in buffer 131 that corresponds to a 60 
vector of Is and 0s in buffer 135, determines the number 
of Is in that buffer 135 vector, and tentatively selects 
from codebook 136, one at a time, those vectors with an 
equal number of Is. It then evaluates the combined 
energy of all coefficients that would be transmitted if 65 
the tentatively selected vector were finally selected, 
and does finally select that tentatively selected vector 
that would transmit the set of coefficients with the larg- 



est combined energy. The selected codebook vector 
and the selected coefficients from buffer 131 are applied 
to packing block 140. 

The function of block 140 is to sort the information 
by the degree of importance that the sorted information 
has for the development of a high quality reproduction 
of the video frame, and to transmit as much of the im- 
portant information as is possible within the constraints 
of the channel capacity. This packing function is 
achieved in block 140 by creating a table that contains 
four columns: a codebook vector identifier column, a 
number of selected coefficients column, a block number 
identifier column, and an importance measure column 
(e.g. total energy in the subbands selected for transmis- 
sion in accordance with the l's in the vector). As an 
aside, the second column, which indicates a number of 
selected coefficients, is strictly a function of the code- 
book. For example, for a codebook of 16 codebook 
vectors, the vector identifier can be a 4-bit number in 
the range 0000 to 1111. Vector identifier 0000 may 
correspond, for example, to vector 01001 10001 1 10000. 
That means that whenever vector identifier 0000 is 
found in the first column of the table, the second col- 
umn of the table contains the number 48 (which is the 
number of l's in vector 0100110001110000 times 8—8 
being the number of pixels in the subband group). 

The packing process proceeds in block 140 by sorting 
on the "importance" column. As illustrated in the table 
below, the first entry belongs to codebook vector 1001, 
with 56 selected error coefficients. This entry is for 
block 23, which has an energy level of 731. The second 
entry belongs to codebook vector 1100, with 24 se- 
lected error coefficients. This entry is for block 511, 
which has an energy level of 623. The third entry be- 
longs to codebook vector 0001, with 3 (different) se- 
lected subbands. This entry is for block 127, which has 
an energy level of 190. Subsequent entries (which are 
not shown) have lower energy levels. 



vector ID 


# of coeff. 


block ID 


importance 


1001 


56 


23 


731 


1100 


24 


511 


511 


0001 


24 


127 


190 


0101 


8 


1023 


102 



In addition to the above sorting and selection from 
the top of the sorted list, block 140, performs the above 
described signal combining function signals when the 
magnitudes of the error signals so suggest (per FIG. 9). 
By selecting from the sorted table, keeping track of the 
number of selected coefficients (column 2), performing 
the "many to one" and "one to many" mappings and 
augmenting the selections information in accordance 
with these mappings, block 140 is able to keep track of 
channel capacity that is taken up by the entries selected 
from the sorted table. When the alloted capacity (i.e., 
number of PAM pulses) is exhausted, selections from 
the table are terminated. 

The above-described encoding, selecting, and pack- 
ing approach of encoder 130 is merely illustrative, of 
course. FIG. 11 presents another approach. 

In FIG. 11, the input to encoder 130 is applied (as in 
FIG. 10) to buffer 131. The groups of 8 pixels from each 
of the 64 subbands (vector 36) are accessed from buffer 
131 and applied to block 141, which develops a measure 
of the energy in each of the elements of the 64-element 
vector. This information is applied to 256 codebook 
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vector switching circuits 142. Each circuit 142 merely 
passes the energies of the elements that correspond to a 
1 in a codebook vector. Thus, the output of a circuit 142 
that pertains to codebook vector P provides a measure 
of the energy that would be sent if that codebook vector 5 
were used. On first blush, one might believe that send- 
ing the most energy is best, and that would suggest 
selecting a codebook vector with a large number of Is. 
However, an unadulterated measure of the energy may 
be counter indicated. Since the total number of Is in the 10 
selected codebook vectors is fixed, a better approach 
would be to maximize the benefit that each 1 in the 
selected vectors provides. Selecting the latter measure, 
each switch 142 in FIG. 11 is followed by a benefit 
measuring circuit 143. The benefit measure may be the 15 
average energy 1 in the codebook vector, or some other 
measure. 

To optimize the selection, the outputs of the 256 
benefit measuring circuits are applied to selector 144. It 
selects the codebook that offers the greatest benefit per 20 
1 in the codebook; which is also the greatest benefit per 
transmitted set of 8 prediction error coefficients. The 
output of selector 144 is applied to packing block 149, 
which is very similar to packing block 140. To wit, 
block 149 sorts the chosen vectors by their benefit mea- 25 
sures selects from the top of the sorted list performs the 
"many to one" and "one to many" mappings as appro- 
priate, and accumulates signals to be transmitted until 
the transmission capacity is exhausted. 

The above-described concept of packing more than 30 
one PAM pulse in a single slot improves performance 
via a better utilization of the available capacity as it is 
reflected by area 99 of FIG. 8. There is another aspect 
of area 99 that may be addressed, and that is noise im- 
munity. Since noise that is introduced by the transmis- 35 
sion medium (between the transmitter and the receiver) 
is independent of the level of the transmitted signal, it is 
clearly advisable to transmit as large a signal as possible. 
This can be achieved in the system of out invention by 
introducing a controllable gain feature (CGF) into the 40 
transmitter and receiver. 

As suggested above, the "one to many" signal map- 
pings process interacts well with the CGF process be- 
cause large signals are broken up into intermediate sig- 
nals, that reduces the overall "spikiness" of the signal. 45 
The lack of very large signals permits a larger CGF 
signal to be applied to element 154, and that results in a 
greater portion of area 98 to be encompassed by the 
signal energy. 

Because of the noise that inevitably is introduced in 50 
the course of transmitting information to the receiver, 
one has to expect that, in spite of the CGF action, the 
information within buffer 165 will eventually differ 
from the information in buffer 160. This problem is 
overcome by the well known technique of inserting in 55 
the transmitted error signal a fraction of the true image, 
and by discarding in the receiver a corresponding frac- 
tion of the image stored in buffer 165. This is known as 
"signal leak**. 

FIGS. 12 and 13 present block diagrams of a transmit- 60 
ter and a receiver that include signal leak and CGF. The 
CGF capability is achieved by applying the PAM 
pulses delivered by block 140 (within block 130) to a 
buffer 152, which basically is a delay line. Processor 
153, which is also responsive to the output of block 140 65 
determines, at fixed intervals, the largest PAM pulse 
within the delay line buffer. Based on that information, 
a multiplicative factor is selected and applied to multi- 



plication element 154, which receives its second input 
from buffer 152. The result is sent to formatter 170. The 
multiplicative factor, which is the CGF control signal, 
is also sent to formatter 170, for transmission to the 
receiver. The CGF action is accounted for in divider 
151 which is responsive to the multiplication factors of 
processor 153 and to the output of element 154. The 
result is sent to decoder 159 which, with the vector 
information of encoder 130 recreates the frequency 
components of transform circuit 120 (as best it can). 
Those frequency components are transformed in block 
161, to reverse the transformation effected in block 120, 
and the results are applied to adder 180. Concurrently, 
the motion vectors of block 110 are applied to translator 
150. With the aid of these vectors, translator 150 modi- 
fies the information of frame buffer 160, and applies the 
results to adder 180. The sum signals developed in adder 
180 are stored in buffer 160. 

Actually, translator 150 is not connected to buffer 160 
directly. Interposed between the two is a divider 189. 
The function of divider 189 is to account for the signal 
leak of block 156. Block 156 is responsive to the frame 
information applied to block 110. It transmits to its 
output a fraction of the signal applied to its input. That 
fraction is added by adder 157 to the motion compen- 
sated signal that is delivered by block 110 to block 120. 
Adder 157 is interposed between blocks 110 and 120. 
The fraction that we use in block 156 is 1/32. Divider 
189, which accounts for the signal leak, also transmits to 
its output a fraction of its input. When the fraction in 
block 156 is 1/32, the fraction in block 189 is 31/32.. 

The receiver of FIG. 13 comports with the transmit- 
ter of FIG. 12. Separator 190 includes means for bulling 
out the LL coefficients, the motion vectors developed 
in block 110 of the transmitter, the prediction error 
vectors of codebook 136 (which may include the map- 
pings information), and the CGF multiplication factors. 
The CGF multiplication factors are applied to a divider 
circuit 158 which complements the actions of circuit 
154 in FIG. 12. The output of divider 158 is applied to 
decoder circuit 163 and the output of circuit 163 is 
applied to transform circuit 155. The output of circuit is 
applied to adder 185 which feeds buffer 165. The output 
buffer 165 is applied to divider block 166, and it supplies 
signals to translator circuit 164. Translator 164 is also 
responsive to the motion vector of block 190 and its 
output forms the second input of adder 185. The func- 
tions of blocks 158, 163, 155 t 185, 165, 166, and 164 are 
identical to the function of blocks 151, 159, 161, 180, 
160, 189 and 150, respectively. As in the receiver of 
FIG. 3, output of buffer 165, which is the received LL 
frame, is applied to adder 186 where it is added to the 
LL frame developed in transform circuit 188. It may be 
noted in passing that divider circuit 158 affects only the 
amplitude of the prediction error signals. Skilled arti- 
sans would surely realize that amplitude of the follow 
decoder 163, rather than precede it, if so desired. 

Although the above-described approach sends the 
motion-compensated prediction error signals in PAM 
format, the principles of this invention are applicable 
with equal efficacy to other modes of transmissions. 
Specifically, experimental results suggest that ex- 
tremely good results can be obtained by sending only 
200,000 error signals. Clearly, these error signals can be 
coded digitally and transmitted in that fashion over 
whatever transmission medium can handle the resulting 
bandwidth. Sending this information digitally over ca- 
ble, for example, would obviate the need for the entire 
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RF section of the transmitter (which is not- shown in 
FIG. 1 anyway) and the RF receiver section. Also, 
organization of the signals which puts the prediction 
error signals in the line scans period and the vector 
information in the retrace period need not be adhered 5 
to. 

The above description presents the principles of this 
invention in the course of describing a transmitter and 
receiver arrangement that is suitable for HDTV. The 
details of construction of the illustrative embodiments 10 
that were presented are not delved into for the sake of 
brevity. All of the blocks that make up the designs 
presented in the figures can be created with conven- 
tional designs without undue experimentation. Indeed, 
many of the blocks in the transmitter and the receiver 15 
perform identical functions and can be constructed in an 
identical manner with conventional components. 

It should be noted that other embodiments can be 
created that are encompassed within the spirit and 
scope of this invention. For example, it has been con- 20 
eluded through experimentation that a better perfor- 
mance is obtained (subjectively) by transforming the 
error signal developed in block 110 (in block 120) and 
by discarding some of the resulting frequency coeffici- 
ents (in block 130). Actually, even in the time domain 25 
the error signal is generally small. When the available 
bandwidth is large and/or when the encoding process is 
efficient enough, it is possible to consider encoding the 
error signals themselves. The really small errors would 
be ignored, the large errors would be encoded, and 30 
some averaging can even be included. For example, the 
pixel that is ignored (because it error level is too low) 
but which is next to a pixel that is selected and encoded 
can be assumed to have a value just under the threshold 
level. Eliminating the need to transform into the fre- 35 
quency domain and back to the time domain clearly has 
a positive effect on the cost of the transmitter and the 
receiver. 

I claim: 

1. A receiver for displaying a video frame in response 40 
to a signal having a retrace interval, a vertical sync 
pulse associated with said retrace interval, and a line 
scans interval timed with respect to said sync pulse, 
comprising: 

first means for receiving prediction error signals dur- 45 
ing said line scans interval; 

second means for receiving motion vector signals, 
during said retrace interval; 

third means responsive to said first means to said 
second means and to video frame information 50 
stored in said third means for developing new 
video frame information to be stored in said third 
means. 

2. The receiver of claim 1 wherein said prediction 
error signals are motion-compensated prediction error 55 
signals. 

3. The receiver of claim 1 wherein said prediction 
error signals are motion-compensated prediction error 
frequency band coefficient signals. 

4. The receiver of claim 1 further comprising vide 60 
display means responsive to said fourth means. 

5. The receiver of claim 1 wherein said second means 
includes 

means for receiving during said retrace interval LL 
coefficients representing low frequency compo- 65 
nents of said video frame, and 

said fourth means combines said LL coefficients with 
said output signals of said third means. 
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6. The receiver of claim 1 wherein said second means 
includes means for receiving during said retrace inter- 
val audio information associated with said frame, and 
said receiver further comprises means for converting 
said audio information to sound. 

7. The method of claim 1 wherein said second means 
includes means for receiving during said retrace inter- 
val error signals encoding information associated with 
said prediction error signals. 

8. A receiver for displaying a video frame in response 
to a signal having a retrace interval, a vertical sync 
pulse associated with said retrace interval, and a line 
scans interval timed with respect to said sync pulse, 
comprising: 

means for receiving prediction error signals during 
said line scans interval and for receiving motion 
vector signals during said retrace interval; 

a frame buffer for storing a current set of signals 
representing at least a portion of said video frame; 

translator means for combining said current set of 
signals stored in said frame buffer with said motion 
vectors; 

means for augmenting output signals of said transla- 
tor means in response to said prediction error sig- 
nals received by said means for receiving; and 

means for storing output signals of said means for 
augmenting in said frame buffer. 

9. The receiver of claim 8 wherein said means for 
receiving includes means for receiving LL coefficients 
representing two-dimensional low frequency compo- 
nents of said video frame, and said receiver further 
comprises means for converting said LL coefficients to 
time domain signals and means for adding said time 
domain signals to with output signals of said buffer to 
develop said video frame. 

10. The receiver of claim 8 wherein said means for 
receiving includes means for receiving error signals 
encoding information, and said receiver further com- 
prises combining means interposed between said means 
for receiving and said means for augmenting, for com- 
bining said prediction error signals and said error sig- 
nals encoding information. 

11. The receiver of claim 10 wherein said prediction 
error signals represent spatial frequency coefficients 
and said augmenting means includes transform means 
for converting the applied prediction error signals to 
spatial domain signals. 

12. The receiver of claim 10 wherein said error sig- 
nals coding information comprises coded information to 
identify subbands contained in said prediction error 
signals. 

13. The receiver of claim 8 wherein said prediction 
error signals represent spatial frequency coefficients 
and said augmenting means includes transform means 
for converting the applied prediction error signals to 
spatial domain signals. 

14. The receiver of claim 8 wherein between said 
means for receiving and said means for augmenting 
means are included for modifying the amplitude of the 
prediction error signals applied to said means for aug- 
menting. 

15. The receiver of claim 8 where said translator 
means combines a fraction of said current set of signals 
stored in said frame buffer with said motion vectors, 
where said fraction is less than 1. 

16. The receiver of claim 8 wherein said means for 
receiving includes 
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means for receiving digital information indicative of 
the position and amplitude of missing samples in 
the prediction error signals received by said means 
for receiving; 

means for developing said missing samples; and S 
means for injecting the developed missing samples in 
said prediction error signals, 

17. The receiver of claim 8 further comprising de- 
coder means, interposed between said means for receiv- 
ing prediction error signals and said means for augment- 10 
ing, for mapping a single error signal unit into a plural- 
ity of error signal units. 

18. The receiver of claim 8 further comprising de- 
coder means, interposed between said means for receiv- 
ing prediction error signals and said means for augment- IS 
ing, for mapping a plurality of error signal units into a 
single error signal units. 

19. The receiver of claim 8, wherein said means for 
receiving includes means for receiving error signal 
mapping control signals and said receiver further com- 20 
prising decoder means, interposed between said means 
for receiving prediction error signals and said means for 
augmenting, for mapping a single error signal unit into 

a plurality of error signal units and a plurality of error 
signal units into a single error signal unit, based on said 25 
mapping control signals. 

20. The receiver of claim 8, wherein said means for 
receiving includes means for receiving error signal gain 
control signals and said receiver further comprising 
gain modification means, interposed between said 30 
means for receiving prediction error signals and said 
means for augmenting, for modifying amplitudes of said 
prediction error signals in accordance with said gain 
control signals. 

21. The receiver of claim 8 wherein said means for 35 
augmenting includes means for transforming said pre- 
diction error signals from the spatial frequency domain 

to the spatial domain. 

22. A method for developing a video transmission 
signal representing an applied video frame signal, said 40 
video transmission signal having a retrace interval, a 
vertical sync pulse associated with said retrace interval, 
and a line scans interval timed with respect to said sync 
pulse, comprising the steps of: 

developing motion vectors from comparison of said 45 
applied video frame signal and an applied video 
frame signal of the immediately previous frame; 

developing prediction error signals from a compari- 
son of said applied video frame signal and said 
. applied video signal of the immediately previous 50 
frame modified by said motion vectors; 

encoding said prediction error signals to form en- 
coded prediction error signals; and 

formation said video transmission signal, with said 
encoded prediction error signals in said line scans 55 
intervals and said motion vectors in said retrace 
interval. 

23. The method of claim 22 wherein said step of en- 
coding includes a step of mapping a number of predic- 
tion error signals to one prediction error signal. 60 

24. The method of claim 22 wherein said step of en- 
coding includes a step of mapping one prediction error 
signal to a number of prediction error signals. 

25. The method of claim 22 wherein said step of for- 
matting including a step of selecting less than all of said 65 
encoded prediction error signals. 

26. The method of claim 25 wherein said step of se- 
lecting selects said encoded prediction error signals 



based on a chosen importance parameter and selecting 
those of said encoded prediction errors signals whose 
importance parameter exceeds a predetermined thresh- 
old. 

27. The method of claim 26 wherein said importance 
parameter is indicative of the contribution of the predic- 
tion error signals to an authentic recreating of said ap- 
plied video frame. 

28. The method of claim 27 wherein the selecting in 
said step of selecting is terminated when a given number 
of prediction error signals is selected. 

29. A method for developing a video transmission 
signal representing an applied video frame signal, said 
video transmission signal having a retrace interval, a 
vertical sync pulse associated with said retrace interval, 
and a line scans interval timed with respect to said sync 
pulse, comprising the steps of: 

developing LL coefficients that represents two-di- 
mensional low frequency characteristics of said 
applied video frame signal; 

developing an LL image represented by said LL 
coefficients; 

developing an LL image signal by subtracting said 
LL image from said applied video frame image; 

developing motion vectors from comparison of said 
LL image signal and an LL image signal of the 
immediately previous frame; 

developing prediction error signals from a compari- 
son of said LL image signal and said LL image 
signal of the immediately previous frame modified 
by said motion vectors; 

encoding said prediction error signals to form en- 
coded" prediction error signals; and 

formatting said video transmission signal, with said 
encoded prediction error signals in said line scans 
interval, and said motion vectors and said LL coef- 
ficients in said retrace interval. 

30. The method of claim 29 further comprising the 
steps of: 

decoding said encoded prediction error signals to 
form estimated receiver prediction error signals; 

translating estimated video frame signals stored in a 
frame buffer in accordance with said motion vec- 
tors to form motion compensated signals; 

combining said motion compensated signals with said 
estimated receiver prediction errors to form new 
estimated video frame signals; and 

storing said estimated video frame signals in said 
buffer. 

31. The method of claim 29 wherein said encoding 
comprises the steps of: 

developing N band-pass signal representations of said 
prediction error signals 

dividing said N band-pass signal representations into 
M error signals vectors, where each error signals 
vector had N members and each member includes 
at least one prediction error signal from a selected 
one of said N band-pass signal representations; and 

for each of said M error signals vectors, selecting a 
codebook vector from a given set of codebook 
vectors that best approximates said error signals 
vector. 

32. The method of claim 31 further comprising the 
steps of: 

formatting at least a subset of said selected codebook 

vectors in said retract interval, and 
formatting the error signal vectors associated with 

said formatted subset of codebook vectors in said 
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line scans interval, with said subset selected to 34. The method of claim 29 further comprising a step 

. . . „ . , _ . of*Wny to one" mapping of said encoded error predic- 

mclude a sufficient number of said error signal t - Qn ^g^^ 

vectors to essentially fill but not exceed said line 35. The method of claim 29 further comprising a step 

scans interval. 5 of "many to one" mapping and "one to many" mapping 

33. The method of claim 32 further comprising a step of said encoded error prediction signals. 

J . * , , . 36. The method of claim 29 further comprising a step 

of ordering the selected codebook vectors and their of amplitude modification error prediction signals in 

associated error signal vectors by a chosen criterion to said line scans interval. 

form an ordered set, with the portion of selected code- 1° 3?* method of claim 29 further comprising a step 

book vectors and associated error signal vectors chosen of .^on a fraction of said applied video image into 

. said prediction error signals in the course of said devel- 

for formatting being taken from the top of said ordered oping prediction error signals, 

set. ♦ * * ♦ + 
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